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The experiments described in this paper are part of a larger joint MIT/NASA research effort and *focus on the 
development of a methodology for designing and evaluating integrated interfaces for highly dexterous and multi- 
functional telerobots. Specifically, a telerobotic workstation is being designed for an Extravehicular Activity (EVA) 
anthropomorphic space station telerobot called Robonaut. Previous researchers have designed telerobotic 
workstations based upon performance of discrete subsets of tasks (for example, peg-in-hole, tracking, etc.) without 
regard for transitions that operators go through between tasks performed sequentially in the context of larger 
integrated tasks. The experiments presented here took an integrated approach to describing teleoperator 
performance and assessed how subjects operating a full-immersion telerobot perform during fine position and gross 
position tasks. In addition, A Robonaut simulation was also developed as part of this research effort, and 
experimentally tested against Robonaut itself to determine its utility. Results show that subject performance of 
teleoperated tasks using both Robonaut and the simulation are virtually identical, with no significant difference 
between the two. These results indicate that the simulation can be utilized as both a Robonaut training tool, and as a 
powerful design platform for telepresence displays and aids. 

INTRODUCTION 

Research is currently being conducted to design and test an intuitive and synthesized telerobotic workstation 
configuration for controlling a high degree of freedom dexterous manipulator for use on die International Space 
Station. The construction and maintenance of the International Space Station is expected to increase the number of 
Extravehicular Activity (EVA) hours by a factor of four over current Space Shutde missions, resulting in higher 
demands on the EVA crewmembers and EVA crew systems. One approach to utilizing EVA resources more 
effectively while increasing crew safety and efficiency is to perform routine and high-risk EVA tasks telerobotically. 
In response, NASA’s Johnson Space Center (JSC) is developing an anthropomorphic telerobot called Robonaut (see 
Figures 1,2) that is capable of performing all of the tasks required of an EVA suited crewmember. 


* Copyright © 2001 by Jennifer Rochlis. Published by the American Institute of Aeronautics and Astronautics, Inc. 
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Figure 1. Robonaut (in development). 


Robonaut is comparable in size to a suited crewmember and requires the operator to command over 46 degrees of 
freedom while performing full immersion telerobotic tasks. The desire to develop a methodology for designing 
integrated workstations is motivated by next-generation robots such as Robonaut. The current robotic workstation 
for the Space Station robots consists of flat panel displays and 6 DOF hand controllers. This is insufficient for 
controlling highly dexterous anthropomorphic manipulators such as Robonaut. The workstation must be designed to 
allow an operator to intuitively control numerous degrees of freedom simultaneously, in varying levels of 
supervisory control, for all types of EVA tasks. 



Figure 2. Graphic of Robonaut performing an ISS EVA task 


Great amounts of research have been conducted in human factors areas such as telerobotic interlaces, human- 
machine interactions, and sensory substitution. However, many of the tasks performed in the experiments described 
in the literature do not capture the variety and complexity of the tasks required of an EVA crewmember. In most 
studies, optimal workstation components are determined based on performance of discretized subtasks (such as peg- 
in-hole, tracking, target acquisition, etc.) without regard to the transitions that the operator must go through between 
tasks performed sequentially (Burdea, 1996; Cannon and Thomas, 1997; Massimino, 1992; Sheridan, 1994). In 
addition, much of the research focuses on a particular hardware or software aspect of the workstation without 
addressing the synthesis of components required to tackle the human factors and controls issues of the system as a 
whole (Kazerooni and Snyder, 1995; Liu and Tharp, 1993; Massimino, 1988; Patrick, 1990; Shattuck, 1994; 
Sheridan, 1993; Vidov, 1993). Finally, the few groups that have looked at workstations as a whole either have not 
had to control as many degrees-of- freedom as Robonaut demands, or have controlled high degree-of-freedom robots 
that lack die dexterity of Robonaut, and therefore employ hand controllers (Akin, 1986; Homan and Gott, 1996; Li 
and Cox, 1995; Sheridan, 1992; Tachi, 1991). 

A concern when designing workstations for robots such as Robonaut is what method should be used for 
development of new interfaces, displays and aids. It is safer and more effective to refine such situation awareness 
displays and aids before applying them to the Robonaut hardware, however one must ensure that die methods used 
to develop them are transferable to the actual robot hardware. Finally, the question arises of how to train operators 
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to use the robot. It is desirable to create a library of knowledge and experience for any new operator before allowing 
them to command the robot hardware directly. 

A series of experiments has been devised and conducted at JSC to characterize the effects of telepresence hardware, 
sensory feedback degradation and task integration on full-immersion telerobotic task performance and workstation 
design. In addition, a Robonaut simulation has been developed and evaluated for use as a potentially powerful 
situation awareness development and operator training tool. One hypothesis tested is that subject teleoperation task 
performance using the simulation is comparable to Robonaut teleoperation performance, and therefore the 
simulation can be utilized as a telepresence interface development and operator training platform. 


ROBONAUT TELEOPERATION 

A Robonaut teleoperator wears a variety of virtual reality display and control technology to immerse them in die 
robot’s workspace, thereby creating a sense of ‘presence’ at the robot worksite. The user’s body position, tracked by 
an array of sensors, is sent as a command to the robot software that in turn generates the robot motions. For die 
Robonaut system, the teleoperator is seated in a remote location wearing instrumented Virtual Technologies, Inc. 
(Palo Alto, CA) Cyber Gloves that measure die displacement and bending of the fingers. A Polhemus FASTRAK® 
(Colchester, VT) system measures the position of the subject’s hands, head, arms and head relative to a fixed 
transmitter. For these experiments, only the right hand/arm, chest and head sensor is udlized. 

Robonaut has two cameras for eyes and the live video feed received from them is sent to a Kaiser Electro Optics, 
Inc. (Carlsbad, CA) ProView 60 helmet-mounted display (HMD) such that the human sees through die HMD what 
the robot sees. A transmitter is also mounted on the helmet so that the mouons of the user’s head are tracked. As the 
operator moves his/her head to the right or left, the robot likewise turns its head. In this way, the human is meant to 
feel that they are immersed and present at the robot site doing the tasks themselves. Figure 3 shows a subject seated 
wearing the telepresence hardware. 



ROBOSIM 
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Robosim (See Figure 4) is under development at the JSC Dexterous Robotics Laboratory. It uses the Interactive 
Graphics, Operations and Analysis laboratory (IGOAL) Enigma modeling software (Houston, TX) to create the 
robot models, environment conditions and camera view, and Real Time Innovations, Inc. (RTI) Network Data 
Delivery Service (NDDS) software for developing the necessary communication networks and protocols. 



Figure 4 Robonaut and Robosim graphic 


Robosim employs the identical forward and inverse kinematics as the Robonaut brainstem, therefore given the same 
command signal, the resultant motion of the simulated robot will match that of the Robonaut. Currently, the 
simulation is limited in that it does not model contact forces, therefore is not possible to study grasping and tool 
handling tasks. 

Two Dell Latitude C600 laptops generate the 3-D Robosim views of the robot arms and task panels. Recall that 
Robonaut has two cameras, one for each eye, which together provide stereo vision to the operator. To generate 
stereo vision with an HMD using Robosim, it is necessary to generate two different graphical views of the same 
scene, separated by the same interoccular spacing as the Robonaut cameras. Note that the HMD view generated 
through the simulation has the identical 60 degree diagonal field of view as the Robonaut cameras. Figure 5 shows 
the view from one eye that the subject sees in the HMD. 
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Figure 5 View of right eye through HMD of Robosim 


EXPERIMENTAL METHODS 

A protocol has been developed which tests task time, workload (both objective and subjective) and accuracy for 
gross position, and fine position tasks performed individually. The tasks span the workspace of the robot arm in all 
directions. Examples include swinging the arm through a large range of motion form an initial to final position, and 
fine positioning of the arm utilizing single joints or multiple joints in different orientations. This work was based in 
part, on teleoperation experiments previously conducted by the author. It was shown that up to thirty percent of the 
total task time was spent gaining better situation awareness (SA) (scanning the worksite between movements or 
tasks for a greater sense of the workspace layout and their position within it, and to decide how best to perform the 
next task) and the average across all operators and days was 10%. This work revealed two important observations, 
1) as operator experience with teleoperation increases the time spent gaining SA information does not decrease 
without bound, there exists a baseline amount of SA time required for a given task and workstation configuration; 2) 
this baseline amount of SA time may be reduced by designing appropriate workstation interface aids. The current 
experimental methodology has been designed to both a) isolate the effects of interest and b) minimize the effect of 
confounding variables. This study is likewise designed to not only describe teleoperator performance using 
Robonaut, but to evaluate the newly developed simulation. 

Basis Task Testing 

The basis tasks were devised in order to describe teleoperator behavior during fine position and gross position tasks. 
They were so named after an initial study revealed that all astronaut motions during an EVA operation could be 
categorized as movements of gross position, fine position, grasping or combination of the three. In addition, the 
basis tasks are designed for simplicity and do not require force sensing or force feedback (although could be 
augmented with such). The basis tasks can be completed and compared across a variety of modalities, including 
zero-G, and performed using almost any teleoperated robots or robotic manipulator. 

The basis tasks are comprised of two task panels (see Figure 6), one similar to a FITT tapping task, and the other 
containing a tracing pattern. While each panel combines elements of both fine and gross position movements, the 
tapping task is primarily a fine position task while the tracing task examines gross position movements. 
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Figure 6 Basis task panels 


On the tapping panel, subjects will be instructed to tap between like colors with their index finger. The size of each 
target is one-half inch square (the approximate width of the Robonaut index fingertip) and they are arranged in both 
the horizontal and vertical directions. The white target in the center is the starting point for each trial. The red and 
green targets are one inch from center, orange and blue are three inches from center, and yellow and purple are live 
inches from center. 

The tracing panel involves following a path around the square and through the diagonals with the index linger. The 
clockwise path traces the red-orange-yellow-green-blue (top right to bottom left)-purple (top left to bottom right) 
path, and the counterclockwise path begins with purple (bottom right to top left) and goes in the reverse order. The 
blue and purple lines were oriented such that subject must reposition their hand and arm before tracing those lines. 
Each line is ten inches long and one-half inch wide. The area of the tracing square and the maximum distance from 
the center to the yellow and purple targets was chosen to comply with the reach envelope of the right arm of the 
Robonaut. 

As is mentioned in the previous chapter, the Robonaut simulation was created with future Robonaut training in 
mind. To this end, it was desired to test if the simulation performance could match the robot performance as it was 
tasked to do. As these are the first set of experiments conducted using the simulation, its ability to match 
Robonaut’ s performance had not been quantified. 

The basis tasks were tested in three modes. First, in order to describe the baseline performance for each subject and 
identify their particular subject effects, the basis tasks will be performed manually. In addition, the basis tasks will 
also be performed using a Robonaut simulation and telerobotically, where the subjects command the robot to do the 
grasping, fine position and gross position tasks. 

• Manually with HMD 

To isolate the effect on basis task performance of the vision system hardware, the protocol will be performed 
manually with the addition of the HMD. This will reveal the effect of degraded field of view and depth perception. 

• Telerobotically with Simulation 

Subjects will perform the basis tasks by teleoperating a simulation of Robonaut. The computer generated Robonaut 
will be commanded by the subject and the view from the simulated Robonaut eyes will be displayed to the subject 
through a helmet-mounted display. These tests will act as a training buffer between the manual and the telerobotic 
tasks, as the simulation possesses the same kinematics as the robot and therefore subject will gain practice with the 
kinematics before controlling the robot directly. 
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• Fully Immersed 

To quantify any coupled effects between die vision system and the proprioceptive system, die basis tasks will be 
tested while the subject is fully immersed. They will again wear an HMD displaying die view from the Robonaut 
camera eyes. 

Note that ensuring that the subject wears an HMD for each modality removes the vision system as a parameter in the 
teleoperation description. 

A total of eight subjects (four male and four female) participated in the experiment. None of die subjects had prior 
experience teleoperating Robonaut. For this reason, one hand-one arm tasks were chosen to minimize the number of 
degrees of freedom and therefore complexity of the teleoperation task. This also allowed for greater control over 
Robonaut safety during the trials. All tasks were conducted using die right hand and only right-handed subjects are 
used. 

For each modality (manual, simulated and robotic), a session consisted of 32 trials. There are six colored pairs of 
tapping targets and two tracing directions (clockwise and counterclockwise). Each is performed four times in a 
balanced order. Each trial is 25 seconds in duration. Training sessions were conducted for each modality. Subjects 
were introduced to the specific modality and given the task instructions. For die manual tasks, subjects are 
instructed to tap between like color pairs, or trace the pattern continually until time is called. For die simulated 
tasks, subjects are instructed to do the same however are told additionally not to penetrate die virtual task board with 
either their index finger or their hand (as contact forces are not modeled, subjects may drive the virtual robot hand 
through the plane of the virtual task board). Likewise for the robot trials, subjects are instructed not to “punch” the 
board or drag the robot finger along the board. For the latter two experiments, there was no force feedback to the 
operator as to whether contact was made, however subjects could visually observe if any part of the hand went 
though the virtual task board, and the deflection of the task panel if die robot was in contact with it. Following the 
trials, a subjective questionnaire is administered to the subjects. 


RESULTS AND DISCUSSION 


Table 1 summarizes the repeated measures analysis effects of modality, color (distance from center), location 
(vertical or Horizontal), and gender on number of taps, number of errors. As expected, the number of taps and 
traces completed during the manual trials was greater than with the simulation or robot, however across virtually 
effect, there is no significant difference between telerobotic and simulated telerobotic task performance. All color 
tapping and tracing were similar with the exception of red taps where subjects averaged three more taps than 
telerobotically, enough to make a significant difference (P=0.009) Other areas that did show significance were 
gender effects. Men had significantly more taps dian women for the robotic tasks in all tasks except the clockwise 
trace, however they also had significantly more errors than the female subjects overall (P=0.002). There was no 
statistical difference between horizontal and vertical directions in either number of taps or number of errors. 
However there were significantly more traces in the clockwise direction than in the counter-clockwise direction. 

In conclusion, these experiments have demonstrated that Robonaut telerobotic performance can be equally achieved 
using Robosim and therefore Robosim can be used in the future to develop Robonaut workstation situation 
awareness aids, as well as to develop operator training skills. 
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